1. Field of the Invention
The present invention relates to an image pickup apparatus having a function of suppressing random noise produced by an image pickup element or the like.
2. Description of Related Art
A CCD image sensor, which is an exemplary image pickup element, includes photoelectric conversion elements arranged in a matrix, a transfer unit that transfers charge output from the photoelectric conversion elements, and an amplifier that amplifies the voltage of the charge transferred by the transfer unit and outputs an image signal.
Currently, with image pickup elements widely used in digital still cameras and video cameras, the amount of charge that can be stored in a single photoelectric conversion element is decreasing due to increased miniaturization and higher pixel counts. While pixel sensitivity drops when the amount of stored charge decreases, sensitivity can be raised by increasing the amplification factor of the amplifier. However, raising the amplification factor of the amplifier also amplifies the noise component contained in the charge output from the photoelectric conversion elements. The amplified noise component appears as random noise in the image, and degrades the signal-to-noise (S/N) ratio of the image. Hereinafter, techniques for suppressing random noise will be described.
[1. Low Pass Filtering using a Two-Dimensional Filter]
A variety of techniques already have been proposed for suppressing random noise. For example, patent document 1 (JP H11-41491A) discloses a method for suppressing random noise by performing low pass filtering (LPF) using a two-dimensional filter.
FIG. 16 shows the configuration of a random noise suppression circuit using a two-dimensional filter. When image data is input from an input terminal 41, a synchronization unit 42 synchronizes the target pixel and the eight pixels above, below and diagonally adjacent thereto (neighboring pixels). The image data synchronized by the synchronization unit 42 is output to subtraction units 431 to 438.
FIG. 17 is a schematic diagram showing the concept behind the synchronization in the synchronization unit 42, with the two-dimensional array of pixels in the image pickup element being shown schematically. Since the neighboring pixels ai−1, j−1, ai−1,j, ai−1,j+1, ai,j−1, ai,j+1, ai+1,j−1, ai+1,j and ai+1,j+1, given a target pixel ai,j, cannot be read at the same time as the target pixel, the neighboring pixels are synchronized with the target pixel after being read out, and are then output to the subtraction units 431 to 438.
In FIG. 16, the subtraction units 431 to 438 calculate the difference between the target pixel and each neighboring pixel, and output the calculated differences to correlation detector units 441 to 448. The correlation detector units 441 to 448 detect whether there is a correlation between the target pixel and each neighboring pixel and output the detection results to a counter 45. A correlation is judged to exist if, for example, the absolute value of a difference output from the subtraction units 431 to 438 is smaller than a threshold. Note that in the following description neighboring pixels judged to be correlated with the target pixel are called “correlated pixels”, and neighboring pixels judged not to be correlated with the target pixel are called “uncorrelated pixels”.
The counter 45 identifies correlated pixels, and notifies information identifying the correlated pixels to a selector 46. The counter 45 also outputs the number of correlated pixels to a number generator 49.
The selector 46 replaces the value of uncorrelated pixels in the output of the synchronization unit 42 with zeros, and outputs the result to an adder 47.
The adder 47 adds the output of the selector 46 and outputs the result to a dividing portion 48.
The number generator 49 adds 1 to the number of correlated pixels output from the counter 45, and outputs the result to the dividing portion 48.
The dividing portion 48 divides the output of the adder 47 by the number output from the number generator 49, and outputs the result to an output terminal 50. Here, since the output of the number generator 49 is the number of pixels added by the adder 47, the output of the dividing portion 48 is the average value of the target pixel and the correlated pixels.
As a result of the above processing, an output is obtained in which random noise of smaller amplitude than the threshold set by the correlation detector units 441 to 448 is suppressed. Also, since low pass filtering using a two-dimensional filter is not performed if the amplitude at the edge of the subject is greater than the threshold set by the correlation detector units 441 to 448, blurring of the edge of the image can be prevented.
[2. Mixed Pixel Reading Method]
Patent document 2 (JP 2006-14075A) discloses a mixed pixel reading method in which pixels are read after being added by an image pickup element.
FIG. 18 is a schematic diagram showing exemplary pixel combinations when reading pixels with the mixed pixel reading method. With this mixed pixel reading method, pixels two removed from pixels 61 to 66 (target pixels) in the vertical, lateral and diagonal directions (eight pixels in all) are added and the resultant values are read. If pixel 61 is the target pixel, for example, the eight neighboring pixels 61a to 61h are added to pixel 61 and the resultant value is then read as the pixel signal of pixel 61. Similar processing is performed on pixels 62 to 66, and the resultant values are read as pixel signals.
A mixed pixel reading method such as this enables an image signal with suppressed random noise to be obtained. Also, since pixels are read by the image pickup element after being decimated, the number of read pixels decreases, enabling high speed reading of pixels.
[3. Cyclic Noise Reduction Method]
Patent document 3 (JP 2001-45334A) discloses a cyclic noise reduction method using frame memory.
FIG. 19 is a block diagram showing an exemplary cyclic noise reduction circuit using frame memory. A subtracter 75 computes the difference between image data input from an input terminal 71 and image data of the previous frame read from a frame memory 79, and outputs the result to an attenuation unit 77. The attenuation unit 77 attenuates the image data output from the subtracter 75 by performing a nonlinear process thereon.
FIG. 20 shows the input/output characteristics of the attenuation unit 77. If the absolute value of the difference between the image data input from an input terminal 71 and the image data of the previous frame is less than a threshold Th, the output of the attenuation unit 77 is attenuated to 50%. If the absolute value of the difference exceeds the threshold Th, the output of the attenuation unit 77 is attenuated by more. If the absolute value of the difference exceeds a threshold Th2, the output of the attenuation unit 77 will be zero.
The image data output from the attenuation unit 77 is input to a subtracter 76. The subtracter 76 subtracts the image data output from the attenuation unit 77 from the image data input to the input terminal 71. The image data output from the subtracter 76 is written to a frame memory 79, as well as being output from an output terminal 72.
That is, if the absolute value of the difference between the image data input to the input terminal 71 and the image data of the previous frame is small, the difference can be judged to be random noise. Consequently, subtracting the random noise component output from the attenuation unit 77 from the image data input to the input terminal 71 with the subtracter 76 enables random noise contained in the image data input to the input terminal 71 to be suppressed.
If the absolute value of the difference between the image data input to the input terminal 71 and the image data of the previous frame is large, it is possible to judge that there is motion in the subject. Consequently, the occurrence of a residual image in the portion where motion occurs is prevented by outputting the image data input to the input terminal 71 from the output unit 72 without modification.
Repeatedly performing the above processing on a plurality of frames of input image data enables image data with suppressed random noise to be obtained.
However, with low pass filtering using a two-dimensional filter as disclosed in patent document 1 or a mixed pixel reading method as disclosed in patent document 2, resolution degradation is a problem.
Also, the random noise suppression effect resulting from low pass filtering using a two-dimensional filter is limited by the number of taps in the filter, thereby requiring more taps to obtain a greater effect. The problem is that increasing the number of taps increases the circuitry size.
Further, with a cyclic noise reduction method as disclosed in patent document 3, not many frames can be read by the image pickup element per unit of time when still images are shot using an image pickup element with a large number of pixels, thereby reducing the quality of moving images.